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(54) Graphic user interface text-based interpolation searching 



(57) The present invention provides an electronic 
display system, method and computer readable medi- 
um that operate to facilitate interactive, expressive, tex- 
tual interpolation by a user. The electronic display sys- 
tem includes: (A) a central processing unit, coupled to 
a system bus; (B) a memory coupled to the system bus 
and having loaded therein an operating system, appli- 
cation programs and computer-executable instructions 
for: (1 ) inserting a desired factor onto a first window; (2) 
inserting node terms onto a second window by, for each 
node term, selecting a desired selection from a plurality 
of predetermined selections; and (3) upon a cursor be- 
ing dragged over the second window to a desired node 
term, additively applying characteristics for the desired 
node term to the desired node term based on a proximity 
of the cursor to the desired node term; (C) a display unit 
coupled to the system bus; (D) a cursor control unit ar- 
ranged to provide signals to control movement of a cur- 
sor on the display unit; and the (E) system bus, for link- 
ing the central processing unit, the display unit, the 
memory unit, and the cursor control unit. 
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Description 

Field of the invention 

5 [0001] The present invention relates generally to devices for computer searching and methods and computer-read- 
able media for implementing the same. More particularly, the present invention relates to using an interactive graphical 
interface for searching. 

Background of the invention 

10 

[0002] Computer systems often utilize displays with icons, pictures, text, pop-up, drop-down menus and other graph- 
ical items. In these systems, individual graphical items or combinations of graphical items are typically shown on the 
display monitor. The graphical item is generally activated by a point and click method in which a cursor controlled by 
a mouse, touch device, pen or trackball is pointed at the item, and a button, keyboard command or activation unit is 
is clicked. 

[0003] Generally, three dimensional character animations are incrementally specified using highly technical interfac- 
es wherein the user adjusts one parameter at a time and then previews the result. The final animation is an immutable 
sequence of rendered frames. At the other end of the spectrum of animation are video games wherein the user, in real- 
time, triggers predefined movements, but is given very little control over the nuances of expression and movement. 

20 [0004] Generally, in many graphics implementations, animation may be performed in three dimensions, but typically 
utilizes a complex process called "rendering" to provide a representation of an image. For example, physical objects 
such as tubes, spheres, rectangles, and the like may be used to "build" other, more visually complex objects. Such 
objects may be rotated, stretched, twisted, or the like. However, such modeling generally requires utilizing modifiers, 
which may include a stack of parametric definitions for each desired geometric subclass, thus requiring a large amount 

25 of storage and a complex retrieval system. 

[0005] Various three dimensional animation systems provide additive displacements as a method of combining par- 
tially defined gestures, facial expressions, or morphing targets. Such targets are generally controlled by bands of sliders. 
[0006] Other related interfaces are the joystick music synthesizers, wherein the joystick is used to control various 
aspects of the timbre and dynamics of the sound. The joystick is moved from a first position to a second position, and 

30 the parameters are dynamically mixed based on the second position of the joystick. 

[0007] In addition to automation, more generally, searches in databases are achieved by using key words or expres- 
sions so that an ordered list may be obtained. However, often the list is quite long, and thus is difficult to sort through 
to obtain the desired information. 

[0008] Thus, there is a need for a device and method for a simplified graphical user interface that the user may utilize 
35 in an interactive fashion to provide relevance weightings for searches to facilitate re-ordering of search data and to 
add nuances of expression and movement in animation. 

Summary of the Invention 

40 [0009] The present invention provides an electronic display system operative to facilitate interactive graphical inter- 
face interpolation of text-based factors by a user. The system includes: a central processing unit, coupled to a system 
bus; a memory unit coupled to the system bus and having loaded therein an operating system, application programs 
and computer-executable instructions for: inserting a desired factor onto a first window; inserting node terms onto a 
second window by, for each node term, selecting a desired selection from a plurality of predetermined selections; and 

45 upon a cursor being dragged over the second window to a desired node term, additively applying characteristics for 
the desired node term to the desired factor based on a proximity of the cursor to the desired dot; a display unit coupled 
to the system bus; a cursor control unit arranged to provide signals to control movement of a cursor on the display 
unit; and the system bus, for linking the central processing unit, the display unit, the memory unit, and the cursor control 
unit. 

50 [0010] The characteristics for the node terms are typically at least one of: query terms in a search query; frequency 
of query terms; distribution of query terms; and components in a computer simulation. The electronic display system 
is generally a computer display system. Inserting node terms may be selected to include combining a plurality of desired 
node terms to form a compound node term. 

[001 1] Where desired, a palette in a third window shown on the display unit may be used for selecting a factor to be 
55 inserted onto the first window. 

[0012] In addition, the present invention may be embodied as a method for facilitating interactive, expressive, three 
dimensional textual interpolation on an electronic display system by a user. The steps of the method include: inserting 
a desired factor onto a first window; inserting node terms onto a second window by, for each node term, selecting a 
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desired selection from a plurality of preselected selections; and dragging a cursor over the second window to a desired 
node term wherein characteristics for the desired node term are additively applied to the desired character/image/ 
query term based on a proximity of the cursor to the desired node term. Generally, the characteristics for the node 
terms may be at least one of: 

5 query terms in a search query; frequency of query terms; distribution of query terms; and components in a computer 
simulation. The electronic display system may be implemented by a computer display system. Inserting node terms 
may include combining a plurality of desired node terms to form a compound node term. A palette in a third window 
displayed on the display unit may be used for selecting a desired factor to be inserted onto the first window. 
[0013] Where desired, the steps of the above-described method may be implemented by computer-executable in- 

10 structions in a computer-readable medium. 

[0014] In another embodiment, the method of the present invention may be implemented by performing the steps 
of: dragging a pointer over an arrangement of a plurality of node terms in a target window wherein placement of each 
node term represents a displacement of a state of an object from a base state; and redrawing or updating the base 
state of the object in a display window in accordance with the proximity of the pointer to the text -based node terms as 

'5 the pointer is dragged over the target window. Positions of the plurality of text-based node terms in the target window 
are typically set by the user, generally using the pointer. Each target window typically has a predetermined area of 
influence that is used to determine, based on a position of the pointer, the displacement to be applied to the object. 
The state of the object may be redrawn or updated by putting an object into a default base state when a position of 
the pointer changes, then applying node terms to the object based on a weighting of each node term, wherein the 

20 weighting is calculated based on the displacement of the pointer from the node term. Where desired, each redrawing 
or updating of the base state of the object may be recorded to provide a textual update path which ma be edited. Where 
selected, multiple node terms with individual weightings may be applied simultaneously. 

[001 5] If desired, the steps of the above-described method may be implemented as computer-executable instructions 
in a computer-readable medium. 



Brief Description of the Drawings 

[0016] The accompanying drawings, which are incorporated into and form a part of the specification, illustrate pre- 
ferred embodiments of the present invention and, together with the description, serve to explain the principles of the 
30 invention. The drawings are only for the purpose of illustrating a preferred embodiment of the invention and are not to 
be construed as limiting the invention. In the drawings, like numbers refer to like parts throughout and mouse clicking 
for a cursor is shown by a circle: 

Figure 1 is a graphical representation of a 4-way split screen depiction of one embodiment of an implementation 
35 scheme for using a graphical interface for animation in accordance with the present invention. 

Figure 2A is an illustration of a character in a dance pose based on clicking a mouse for a cursor very close to the 
sway-right target pose selection shown on the skeleton controller in Figure 2B. 

Figure 3A is an illustration of a character in a dance pose based on clicking a mouse for a cursor approximately 
equidistant from the sway-right target pose selection and the sway-left target pose selection shown on the skeleton 
40 controller in Figure 3B. 

Figure 4A is an illustration of a character in a dance pose based on clicking a mouse for a cursor very close to the 
sway-left target pose selection shown on the skeleton controller in Figure 4B. 

Figure 5A is an illustration of a character in a dance pose based on clicking a mouse for a cursor approximately 
equidistant from the sway-right target pose selection, the inbetween pose selection and the sway-left target pose 
45 selection shown on the skeleton controller in Figure 5B. 

Figure 6A is an illustration of a character in a dance pose based on clicking a mouse for a cursor very close to the 
inbetween target pose selection shown on the skeleton controller in Figure 6B. 

Figure 7A is an illustration of a facial configuration based on clicking a mouse for a cursor in an area apart from 
the target selection shown on the facial controller in Figure 7B. 
so Figure 8A is an illustration of a facial configuration based on clicking a mouse for a cursor in an area substantially 

on the surprise target selection shown on the facial controller in Figure 8B. 

Figure 9A is an illustration of a facial configuration based on clicking a mouse for a cursor in an area substantially 
halfway between the surprise target selection and the anger target selection shown on the facial controller in Figure 
9B. 

55 Figure 1 0A is an illustration of a facial configuration based on clicking a mouse for a cursor in an area substantially 

on the anger target selection shown on the facial controller in Figure 10B. 

Figure 11 A is an illustration of a facial configuration based on clicking a mouse for a cursor in an area substantially 
halfway between the anger target selection and the smile-big target selection shown on the facial controller in 
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Figure 11 B. 

Figure 1 2A is an illustration of a facial configuration based on clicking a mouse for a cursor in an area substantially 
on the smile-big target selection shown on the facial controller in Figure 12B. 

Figure 1 3A is an illustration of a facial configuration based on clicking a mouse for a cursor in an area substantially 
5 equidistant between the anger target selection, the surprise target selection and the smile-big target selection 

shown on the facial controller in Figure 13B. 

Figure 14A is an illustration of a configuration of a series of blocks based on clicking a mouse for a cursor in an 
area apart from the target selection shown on the block controller in Figure 14B. 

Figure 15A is an illustration of a configuration of a series of blocks based on clicking a mouse for a cursor in an 
to area approaching the tine target selection shown on the block controller in Figure 15B. 

Figure 16A is an illustration of a configuration of a series of blocks based on clicking a mouse for a cursor in an 
area substantially on the line target selection shown on the block controller in Figure 16B. 

Figure 17A is an illustration of a configuration of a series of blocks based on clicking a mouse for a cursor lying 
intermediate between the line target selection and the spiral target selection shown on the block controller in Figure 
15 17B. 

Figure 18A is an illustration of a configuration of a series of blocks based on clicking a mouse for a cursor in an 
area substantially on the spiral target selection shown on the block controller in Figure 1 BB. 
Figure 19A is an illustration of a configuration of a series of blocks based on clicking a mouse for a cursor in an 
area substantially on the star 3 target selection shown on the block controller in Figure 19B. 
20 Figure 20A is an illustration of a configuration of a series of blocks based on clicking a mouse for a cursor lying 

intermediate between the line target selection and the star 3 target selection shown on the block controller in Figure 
20B. 

Figure 21 A is an illustration of a configuration of a series of blocks based on clicking a mouse for a cursor lying 
closer to the line target selection than to the star 3 target selection shown on the block controller in Figure 21 B. 
25 Figure 22 is an illustration of a recording of a series of facial expressions utilized wherein the recording may be 

used to re-implement the series. 

Figure 23 illustrates one embodiment of a block diagram of an electronic display system operative to facilitate 
interactive searching by a user in accordance with the present invention. 

Figure 24 shows one embodiment of steps for a method or computer-readable medium having computer-execut- 
30 able instructions for facilitating interactive searching on an electronic display system by a user in accordance with 

the present invention. 

Figure 25 shows one embodiment of steps for a method or computer-readable medium having computer-execut- 
able instructions for facilitating searching using a graphical user interface in accordance with the present invention. 
Figures 26A-26C illustrate one embodiment of a graphic user interface in accordance with the present invention. 

35 

Description of the Preferred Embodiments 

[0017] The present invention provides a graphical user interface which allows the user to interactively change the 
state of an object or objects. The interface is controlled by a pointing device (mouse or tablet) or any other input device 

40 whose output may be translated to an x,y position on a second window. 

[0018] In automation: The visual changes of the object that occur as the pointer is moved create an animation se- 
quence. Since the user may see the resulting change of the object in real-time as the pointer moves, the process of 
creating the animation becomes a truly interactive performance. This is much different than the more step-by-step 
procedures used in traditional animation interfaces. 

45 [001 9] In a search: The interface of the present invention may also be used interactively to search to find a desired 
state of the object. Since the user may see the resulting change of the object as he moves the pointer, the search 
process is guided by a tight feedback loop. In the current art, database queries are delimited through a set of keywords 
or expressions, and the result is displayed as an ordered list of documents that contain the set of keywords or expres- 
sions. The documents are ordered by relevance, where relevance is defined algorithmically as a match between the 

so query terms and the frequency and distribution of those terms, or semantic associations of those terms, in the docu- 
ments. Modifying the query will result in a reordered list. Examples of modifications are specifying different degrees of 
relevance of the terms, deleting some terms, introducing new terms, and indicating which terms should be close to 
one. The current art for describing these modifications includes Boolean strings, binary relevance operators (e.g., + 
or -), checklists, and graphical techniques for inserting or deleting terms. Often a query will return a large set of docu- 

55 ments and it is difficult to determine algorithmically the appropriate relevance weightings that fit the needs of the user. 
In contrast, the present invention provides a novel method for rearranging the query result. Using techniques developed 
for real-time graphical manipulation of animated figures, a graphical user interface may be used in accordance with 
the present invention to rearrange the query result. Each node in a graph represents a term in the query. One advantage 
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provided by the present invention is that the nodes may be arbitrarily rearranged in a two dimensional, or simulated 
three dimensional; display. Thus, terms with no semantic association may be arranged near one another. Positioning 
a cursor in the display results in a unique ordering of the query result. Notably, the cursor need not be on a node or on 
the direct path between nodes. If the cursor is on a node, then the term associated with that node receives the strongest 

5 relevance weighting. As the cursor is repositioned between nodes, the distance between the cursor and each node (i. 
e., term) is used to calculate a new set of relevance weightings. Rearranging the graph creates new possible combi- 
nations of weights. Positioning the cursor within the graph defines a unique combination of weights. The position of 
the cursor may be moved within the graph, and each repositioning causes a new list to be displayed. 
[0020] In its basic configuration, the user interface of the present invention consists of two windows, a display window 

10 and a target (controller) window. The target window displays a two dimensional arrangement of target points where 
each target represents a DISPLACEMENT of the object from some BASE STATE. As the user drags the pointer over 
the target window, the object is updated and redrawn in the display window. The state change in the object is determined 
by the proximity of the pointer to the various targets. 

[0021] The user is allowed to arrange the targets as desired, thus defining the potential search or animation space. 
'5 Each target has an area of influence which is used to determine (based on the mouse position) how much of that 
target's displacement is applied to the object. The WEIGHTING CALCULATION may use any arbitrary function, but is 
typically based on the distance between the pointer and the target. How the target displacements are defined or selected 
is defined by a preselected external interface or procedure. 

[0022] The object's state is changed as follows: every time the pointer position changes, the object is first put into 
20 its default base state. Then, all the targets are applied to the object based on their weighting, which is calculated from 
the mouse position. Often, only a small number of targets will be close enough to the mouse to have any influence. 
The object ("PARAMETER OBJECT") may then either be directly displayed or its set of parameters and values may 
be applied to another "DISPLAY OBJECT" which the user sees. The following Lisp pseudo-code shows what happens 
every time the user moves the pointer: 

25 

(defun PROCESS-NEW-INPUT (window targets parameter-object display- 
object pointer-x pointer-y) 
30 (goto-base-state parameter-object) 

35 



40 
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(loop for target in targets 

for weighting = (compute-weighting target pointer-x pointer-y) 
do 

(add-displacement parameter-object (get-displacement target) 

weighting)) 

;; update the DISPLAY-OBJECT from the PARAMETER-OBJECT (if they 
are the same, this is a no-op) 

(apply-update parameter-object display-object) 
(redisplay window display-object parameter-object)) 
(defun ADD-DISPLACEMENT (object displacement weighting) 
(loop for parameter in (get-parameters object) 
do 

(set-parameter-value object parameter 

(+ (get-parameter-value object parameter) 

(* weighting (get-displacement-values displacement)))))) 

[0023] The OBJECT (or "PARAMETER OBJECT') is anything that may be represented or encoded by a sequence 
of properties and their associated numeric values. It is this object which is directly changed by the interface in the 
manner described in the algorithm. 

[0024] A STATE is a given set of parameters and values associated with a particular object. So, for example, the 
state of a television set might be defined by the values for the current channel, contrast, brightness, color and tint. Or 
the state of a human body, for animation purposes, might be defined by the state of bone rotations around their re- 
spective joints. One set of angles may define a "throw" pose and another may define a "crouch" pose. 
[0025] The DISPLAY OBJECT is the object that the user sees in the display window. The DISPLAY OBJECT poten- 
tially may be different from the parameter object. The translation from the parameter object to the display object may 
be arbitrarily complex. For example, if the parameter object represents various parameters used in a physical simulation 
(gravity, wind velocity, friction, etc.), those parameters will be applied in the simulated environment, and the simulated 
environment will be redisplayed. On the other hand, often the parameter object may be displayed directly itself. For 
example, a human animation skeleton may serve both as a displayable structure and as a container for the various 
parameters and values that define the different body pose states. 

[0026] The BASE STATE of an object is a default state that is used as a reference to save displacements (see below). 
[0027] A DISPLACEMENT is the difference between two states of an object. Typically, a DISPLACEMENT is the 
difference between a desired target state and the base state. 

[0028] The WEIGHTING CALCULATION is the function used to compute how much influence a given target has. 
One simple weighting calculation uses a radial falloff where each target represents the center of a circle. When the 
mouse is directly over the target (in the center of the circle), the full weight of that target's displacement is applied to 
the object. When the mouse is outside of the circle, the target has no effect on the object's state, and the weighting 
inside the circle is a number from 0.0 to 1 .0, reflecting the pointer's distance from the center. 

[0029] It is clear that not all possible states may be reached by this interface given a particular set and arrangement 
of targets. Thus, only those states that result from the linear combination of the defined target displacements, wherein 
the weightings are derived from the mouse position, may be achieved. The present invention provides a new user 
interface technique, system or method that may be applied to various domains. 

[0030] The present invention provides a real-time interface for performer animation that enables a user to create 
various classes of targets with different behaviors, internal structure, and graphical representations wherein the ani- 
mation includes some of the expressive power of scripted systems with natural input gestures and real-time control of 
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interactive systems. In addition, target poses for performer animation may be combined into compound target poses 
and treated as one unit. Thus, the present invention provides the advantages of real-time animation with less complexity 
in providing development f expression and movement for target poses. 

[0031] The present invention may be used to control body movement, dancing and facial expressions of a three 
5 dimensional character in an on-line shared music environment, chat room, interactive game or the like. The invention 
may also be applied to dynamically control sound synthesis. 

[0032] Figure 1 is a graphical representation of a 4-way split screen depiction of one embodiment of an implemen- 
tation scheme on a computer screen in accordance with the present invention. In the upper left portion of Figure 1 , a 
wire-frame depiction of a head 102 is shown as a basic unit to be animated. The basic unit to be animated is selected 

10 by using a cursor directed by a mouse, a mouse-directing post, touchpad or the like, to click on the desired geometry 
selection selected from the plurality of geometry selections shown on the geometry selection screen 104 indicated in 
the lower left portion of Figure 1 . Where desired, the user may select to have the wire frame covered with "skin" that 
may be shaded in various sections. However, it should be noted that selection of the skin-covered version of the head 
requires more memory than the wire-frame version of the head, so that, depending on the computer's computational 

15 capacity, the skin-covered version of the head may move at a slower rate than the wire-frame version of the head. 
Typically, for example, a 450 MHz computer with a Pentium processor may implement the present invention. 
[0033] After selecting the basic unit to be animated, at least one target screen is selected for the upper right portion. 
In the example shown in Figure 1 , target facial expressions of Fear 106, Surprise 108, Anger 110, Surprise Smile 112, 
Smile II 114 and Disgust 116 have been selected by clicking on the + button to show a pull-down screen with available 

20 facial expressions, thus providing a dot target or anchor on the screen for each selected facial expression, and then 
moving the dot target or anchor by mouse-clicking on the dot target or anchor and dragging the dot target or anchor 
to the desired position on the screen. The software programs listed on the pull-down screen for the facial expressions 
have been prepared previously. Typically, the cursor is placed on a dot target or anchor for a desired facial expression, 
and then the cursor is moved to a dot target or anchor for a next desired facial expression. The software for the selected 

25 facial expression interpolates the facial expression as the cursor is moved, animating the head to change from the 
facial expression for the first dot target or anchor to a next facial expression for the next dot target or anchor. Double- 
clicking and dragging the dot targets or anchors closer together, as shown in Figure 1 when the dot target or anchor 
for Anger 110 is dragged to a position 118 close to the Surprise 108, provides for faster change in facial expression to 
the next facial expression when the cursor is moved from Anger 1 8 to Surprise 1 08 or, when the dot targets or anchors 

30 are superimposed, provides for combining the selected facial expressions. As shown in the lower right portion of Figure 
1 , a layer may be added with camera poses. For example, camera pose 1 132 may represent viewing the character 
from a close-up perspective, and camera pose 2 134 may represent viewing the character from a distance. 
[0034] In addition, other layers may be added, as, for example, when the facial expression is to be combined with 
motion. For example, where a wire-frame figure or skin-covered figure is used for a body to be animated, the geometry 

35 may be selected by mouse-clicking in the geometry selection area at the lower left of Figure 1 . A head may be added 
in the same manner, providing, for example, a character shown in Figure 2A. 

[0035] Figure 2A is an illustration of a character 202 in a dance pose based on clicking a mouse for a cursor 204" 
very close to the sway-right 206 target pose selection shown on the skeleton controller in Figure 2B. Two windows are 
utilized: a performer window and a target pose window, wherein the target pose window may have a plurality of layers. 

40 Each layer of the pose window shows available characteristics for a skeletal action, facial expression, or the like. For 
example, in the lower right portion of Figure 1, target poses Arms Up 120, Arms Raised Halfway 122, Sway Left 124, 
Sway Right 1 26 are selected in a manner similarto the selection of facial expressions. Where desired, the target screen 
on the lower right portion of Figure 1 may be selected and added to the target screen on the upper right portion of 
Figure 1 . When the cursor is dragged over the overlaid layers, the facial expression of the character, as well as his 

45 target pose, change in accordance with the position of the cursor relative to the target facial expressions and the target 
poses. 

[0036] In addition, customized poses may be made, for example, by placing the cursor on each desired limb and 
dragging the limb to the desired position. The customized pose is then saved. Figure 3A is an illustration of a character 
302 in a dance pose based on clicking a mouse for a cursor 308 approximately equidistant from the sway-right 304 
50 target pose selection and the sway-left 306 target pose selection shown on the skeleton controller in Figure 3B. As the 
cursor is dragged from one target pose to a next target pose, geometrical x,y values representing the pose in the 
underlying software program are interpolated, and the screen shot illustrated shows a change in the pose from a swing- 
right pose toward a swing-left pose. 

[0037] Figure 4A is an illustration of a character 402 in a dance pose based on clicking a mouse for a cursor 408 
55 very close to the sway-left 406 target pose selection shown on the skeleton controller in Figure 4B. Figure 5A is an 
illustration of a character 502 in a dance pose based on clicking a mouse for a cursor 504 approximately equidistant 
from the sway-right 508 target pose selection, the inbetween 506 pose selection and the sway-left 510 target pose 
selection shown on the skeleton controller in Figure 5B. 
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[0038] Figure 6A is an illustration of a character 602 in a dance pose based on clicking a mouse for a cursor 604 
very close to the inbetween 606 target pose selection shown on the skeleton controller in Figure 6B. Where desired, 
the cursor may be clicked on each limb of the character in the poses when the limb is to be moved, and the limb is 
dragged to the desired position. 
5 [0039] Figure 7 A is an illustration of a facial configuration 702 based on clicking a mouse for a cursor 704 in an area 
apart from the target selection shown on the facial controller in Figure 7B. Figure 8A is an illustration of a facial con- 
figuration 802 based on clicking a mouse for a cursor 804 in an area substantially on the surprise target selection 
shown on the facial controller in Figure 8B. Figure 9A is an illustration of a facial configuration 902 based on clicking 
a mouse for a cursor 906 in an area substantially halfway between the surprise target 904 selection and the anger 
10 target 908 selection shown on the facial controller in Figure 9B. Figure 10A is an illustration of a facial configuration 
1002 based on clicking a mouse for a cursor 1004 in an area substantially on the anger target selection shown on the 
facial controller in Figure 10B. Figure 11 A is an illustration of a facial configuration 1102 based on clicking a mouse for 
a cursor 1106 in an area substantially halfway between the anger target 1104 selection and the smile-big target 1108 
selection shown on the facial controller in Figure 1 1 B. Figure 1 2A is an illustration of a facial configuration 1 202 based 
15 on clicking a mouse for a cursor 1204 in an area substantially on the smile-big target selection shown on the facial 
controller in Figure 12B. Figure 13A is an illustration of a facial configuration 1302 based on clicking a mouse for a 
cursor 1310 in an area substantially equidistant between the anger target 1306 selection, the surprise target 1304 
selection and the smile-big target 1308 selection shown on the facial controller in Figure 13B. Clearly, other facial 
expression selections may be utilized. 
20 [0040] Sequential changes may be made to an arrangement of a plurality of boxes in accordance with the present 
invention. The boxes may, for example, be arranged in a circle, a line, a spiral, every third one located inward, every 
fifth one located inward, randomly, or as a vertical circle inside a horizontal circle. By dragging the cursor to the different 
dot targets or anchors, the arrangements are transformed from a first configuration to a second configuration . By adding 
a camera pose layer, the user may zoom in and out for the configurations. 
25 [0041] Figure 14A is an illustration of a configuration of a series of blocks 1402 based on clicking a mouse for a 
cursor 1404 in an area apart from the target selection shown on the block controller in Figure 14B. Figure 15A is an 
illustration of a configuration of a series of blocks 1502 based on clicking a mouse for a cursor 1504 in an area ap- 
proaching the line target 1506 selection shown on the block controller in Figure 15B. Figure 1 6A is an illustration of a 
configuration of a series of blocks 1602 based on clicking a mouse for a cursor 1604 in an area substantially on the 
30 line target 1606 selection shown on the block controller in Figure 16B. Figure 17A is an illustration of a configuration 
of a series of blocks 1 702 based on clicking a mouse for a cursor 1 706 lying intermediate between the line target 1 708 
selection and the spiral target 1 704 selection shown on the block controller in Figure 17B. Figure 18A is an illustration 
of a configuration of a series of blocks 1 802 based on clicking a mouse for a cursor 1 84 in an area substantially on the 
spiral target 1806 selection shown on the block controller in Figure 18B. Figure 19A is an illustration of a configuration 
35 of a series of blocks 1902 based on clicking a mouse for a cursor 1904 in an area substantially on the star 3 target 
1906 selection shown on the block controller in Figure 19B. Figure 20A is an illustration of a configuration of a series 
of blocks 2002 based on clicking a mouse for a cursor 2006 lying intermediate between the line target 2008 selection 
and the star 3 target 2004 selection shown on the block controller in Figure 20B. Figure 21 A is an illustration of a 
configuration of a series of blocks 21 02 based on clicking a mouse for a cursor 21 04 lying closer to the line target 21 06 
40 selection than to the star 3 target 2108 selection shown on the block controller in Figure 21 B. 

[0042] The user may insert facial and skeletal target poses in a performer window by dragging and dropping a se- 
lected pose from a palette of pre-defined target poses in the target pose window. To animate the character, the user 
drags the mouse over the target pose window. The target poses are additively applied to the character based on 
proximity to the cursor. Thus, for example, a smile may dynamically morph into a frown as the cursor moves between 
45 a smile target and a frown target, or one dance step may transition to another dance step. The exact layout of targets 
defines a graphical lexicon of input gestures with corresponding character motions. Various classes of targets may be 
created with different behaviors, internal structure, and graphical representations. In addition to the target poses on 
the palette, there may be other choices such as "Begin Recording", "End Recording", and "Play Recording". By selecting 
"Begin Recording", the cursor path is recorded, and is shown as a marked path on the computer screen. This recording 
50 may be viewed as a video clip by selecting "Play Recording" on the palette and clicking on the marked path. Figure 
22 is an illustration of a recording 2202 of a series of facial expressions utilized wherein the recording may be used to 
re-implement the series. 

[0043] The present invention may be implemented as an electronic display system, shown in one embodiment in 
block diagram form in Figure 23, operative to facilitate interactive, expressive, three dimensional animation by a user. 
55 The electronic display system includes a central processing unit 2302, coupled to a system bus 808; a memory unit 
2304 coupled to the system bus 2308 and having loaded therein an operating system 2312, application programs 2314 
and computer-executable instructions for interactive graphic user-interface animation or searching 231 6; a display unit 
231 0 coupled to the system bus 2308; a cursor control unit 2306 coupled to the system bus 2308; and the system bus 
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2308, for linking the central processing unit, the display unit, the memory unit, and the cursor control unit. In one 
embodiment, the computer-executable instructions for interactive graphic user-interface animation or searching 2316 
may include: inserting a desired character, image, or query term onto a first window, which may be accomplished, for 
example, by selection from a palette in another window; inserting dot targets, anchors, or node terms onto a second 

5 window by, for each dot target, anchor, or node term, selecting a desired pose or selection from a plurality of prede- 
termined poses, selections, or terms; and upon a cursor being dragged over the second window to a desired dot target, 
anchor, or node term, additively applying characteristics for the desired dot target, anchor, or node term to the desired 
character, image, or query term based on a proximity of the cursor to the desired dot target, anchor, or node term. The 
cursor control unit 806 may be a mouse, a moveable control peg in a keyboard, a touch-sensitive pad, or the like. The 

io characteristics for the dot targets, anchors, or node terms are selectable for the use employed by the user. For example, 
for animation, characteristics may include facial expressions, character poses, or camera positions (for zooming in and 
out with respect to the view). The electronic display system may be implemented by a computer display system. Dot 
targets, anchors, or node terms may be combined by dragging a desired dot target, anchor, or node term onto at least 
one other dot target, anchor, or node term to form a compound dot target, anchor, or node term that has the charac- 

15 teristics of the combined dot targets, anchors or node terms. A palette may be generated by the computer-executable 
instructions and shown in a third window on the display unit. The palette may be used for selecting a desired character, 
image or query term to be inserted onto the first window. 

[0044] Figure 24 shows one embodiment of steps for a method or computer-executable instructions 2408 for facili- 
tating interactive, expressive, three dimensional animation or searching on an electronic display system by a user in 

20 accordance with the present invention. The steps include: inserting 2402 a desired character, image, or query term 
onto a first window; inserting 2404 dot targets, anchors, node terms onto a second window by, for each dot target, 
anchor, or node term, selecting a desired pose, selection, or query term from a plurality of preselected poses, selections, 
or query terms; and dragging 2406 a cursor over the second window to a desired dot target, anchor, or query term 
wherein characteristics for the desired dot target, anchor, or node term are additively applied to the desired character, 

25 image, or query term based on a proximity of the cursor to the desired dot target, anchor, or node term. The charac- 
teristics are as described above. The electronic display system is typically a computer display system. Combinations 
of dot targets, anchors, or node terms are made by dragging one dot target, anchor, or node term onto at least one 
other dot target, anchor, or node term to get combined characteristics (interpolated). A palette may be used to select 
characters, images or query terms as described above. 

30 [0045] Clearly, a computer-readable medium having computer-readable instructions 2408 may be used for imple- 
menting the present invention. The instructions provide for executing the steps of the method of the invention and are 
implemented in the fashion described above. 

[0046] As shown in Figure 25, the present invention may be implemented as a method or computer-readable medium 
2502 having computer-executable instructions for facilitating animation or searching using a graphical user interface. 

35 The method or computer-readable medium include steps or computer-executable instructions that include: dragging 
2504 a pointer over a two/three dimensional arrangement of a plurality of target points, anchors, or node terms in a 
target or controller window wherein each target, anchor or node term represents a displacement of a state of an object 
from a base state; and redrawing or updating 2506 the base state of the object in a display window in accordance with 
the proximity of the pointer to the target points, anchors or node terms as the pointer is dragged over the target window. 

40 Where desired, positions of the plurality of target points, anchors, or node terms in the target or controller window may 
be set by the user. 

[0047] For example, the user may use the pointer to position the plurality of target points, anchors, or node terms, 
either individually or as a group. Typically, each target has a predetermined area of influence that is used to determine, 
based on a position of the pointer, the displacement to be applied to the object. The state of the object is generally 

45 redrawn or updated by putting an object into a default base state when a position of the pointer changes, then applying 
targets, anchors, or node terms to the object based on a weighting of each target, anchor, or node term, wherein the 
weighting is calculated based on the displacement of the pointer from the target, anchor, or node term. Each redrawing 
or updating of the base state of the object may be recorded to provide an animation path or search path which may 
be used to reproduce the series of redrawings or updatings. The animation path or search path is typically editable. 

so [0048] For example, pointing and clicking may be used to select and change or delete any portion of the animation 
or search path. Where desired, multiple targets, anchors or node terms with individual weightings may be applied 
simultaneously. 

[0049] Current search engines use a variety of measures to algorithmically determine whether or not a document 
(or more generally an object) is relevant to a query. Common examples include statistical measures such as term 
55 frequency and the inverse document frequency of the term and its distribution across all documents in the database. 
Documents that are longer, more recent, more popular, more frequently linked to by other documents, and whose title 
or metatags contain some of the query terms are typically judged more relevant than documents lacking these char- 
acteristics. Various algorithms (often proprietary) are used to expand the query terms and weight and combine the 
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relevance ranking factors. The present invention is independent of the particular factors. What is important is that a 
set of factors and normative weights for each factor are defined. 

Table 1 



5 


Example 1(33% 33% 33%): trust psychology web 




1. 


Bulkley Book Rvw.html Relevance: 100% 




2. 


High Trust® Thinking/Leader ship/Teaching & Trust Psychology® Relevance: 1 00% 


10 


3. 


Why Christians Can't Trust Psychology Relevance: 100% 




4. 


Why Christians Can't Trust Psychology Relevance: 100% 




5. 


Recommended Reading: Psychology Relevance: 84% 




Example 2 (50% 33% 17%): trust trust trust psychology psychology web 


15 


1. 


DEEP-ECOLOGY mar97 discussion:: Eco-Psychology & Activism: 




2. 


GIVING TO THE AMERICAN PSYCHOLOGICAL FOUNDATION Relevance: 100% 




3. 


Healthcare Psychology Research Group Relevance: 100% 


20 


4. 


PSYCHOHERESY ABOUNDING Relevance: 100% 




5. 


The Trust Advocate 800 Risk Management Program Relevance: 100% 




Example 3 (50% 17% 33%): trust trust trust psychology web web 




1. 


Moving Web Word: Gaining Users' Trust Relevance: 100% 


25 


2. 


CPA R.A. Burrell's Accounting Resource Guide Relevance: 100% 




3. 


Legal Notices about This Web site Relevance: 100% 




4. 


Web Security: A Matter of Trust, WWW Journal: Relevance: 100% 


30 


5. 


Awesome Web Solutions for Small Business - THaNet Relevance: 100% 




Example 4(17% 50% 33%): trust psychology psychology psychology web web 




1. 


WEB Site for Psychology 101 Relevance: 100% 


35 


2. 


Psych Web list of Psychology Megalists Relevance: 100% 


3. 


Community Psychology Awards Given to Web Sites Relevance: 100% 




4. 


Selected Internet Resources for Psychology Relevance: 100% 




5. 


Social Psychology Network Relevance: 100% 


40 


Example 5 (17% 33% 50%): trust psychology psychology web web web 




1. 


Community Psychology Awards Given to Web Sites Relevance: 100% 




2. 


Psychology Resources on the World Wide Web Relevance: 100% 


45 


3. 


WEB Site for Psychology 101 Relevance: 100% 


4. 


Psych Web list of Psychology Megalists Relevance: 100% 




5. 


About the Web Site Relevance: 100% 



[0050] Current search engines allow limited control over the weights and factors used to determine relevance rank- 
ings. Search engines such as 'http://www. altavista.com' and many other popular engines allow terms to be prefixed 
with a "+" indicating that relevant documents must contain the term, and "-" indicating that relevant documents must 
not contain the query term. A few search engines such as 'http://www.infoseek.com' weigh repeated query terms more 
heavily than term used only once in a query. For example, as shown in Table 1 above, where listings were generated 
by infoseek by varying the occurrences of "trust", "psychology", and "web" in a query, it is clear that the query terms 
"trust", "psychology" and "web" retrieve different sets of document references depending upon how often each term is 
repeated. The query was intended to retrieve documents on the psychology underlying attributed trustworthiness of 
websites. As may be seen from examining Table 1 , many of the retrieved references were not relevant to the search 
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goal. Notably, in the examples shown in Table 1, the same number of document references was retrieved for each 
query. What differs is the relative rank assigned to each. 

[0051] Figures 26A-26C illustrate one embodiment of a graphic user interface in accordance with the present inven- 
tion. Figure 26A shows a selection of anchor terms: "trust", "psychology" and "web". The numbered arrows indicate 

5 different cursor positions. The numbers of the arrows refer to the examples shown in Table 1 . Figure 26B shows a 
control screen for selecting or deleting anchor terms. Figure 26C shows a display of a list of documents references 
and summaries generated for cursor position "2" in Figure 26A. As shown in Figures 26A-26C, in the current invention 
the query terms, any terms in the expanded query (i.e., semantic cohorts of the original query), and other relevancy 
factors may be chosen as anchors in a visual display in accordance with the display properties described earlier for 

10 the animation system. The relative position of the anchors may be modified via drag-and-drop manipulations. The 
existence of an anchor may be manipulated by choosing the factor from a list of factors. Positioning the cursor within 
the display will determine how the anchors are weighted, which in turn will determine the relevance ranking of the 
documents. Notably, the anchors may be the type of relevance ranking factors mentioned above, or in a mega search 
engine, e.g., , http;//www,dogpi le.com', the anchors may be various constituent search engines that are used by the 

15 mega search engine. The weight given to a particular domain or the inclusion of an audio or graphic image may also 
be used. The present invention is not limited to the preceding lists of factors but may accommodate any well-defined 
factor. 



Document 
link 


Doc. title 


Doc. Summary 


Last 
modified 


# of 
links 

referencing 
doc 


#of J 
hyper- 
link in 
doc. 


htto://www 
.ap... 


GIVING TO 
THE 

AMERICAN 
PSYCHO- 
LOGICAL 
FOUNDATION 


Description of the 
types of giving to 
the American 
Psycho- 
logical 
Foundation 


15 Apr 
1999 


240 


6 



























45 

[0052] Table 2, shown above, is an illustration of a database generated in client memory to permit fast regeneration 
of a list of document references and summaries. Double-boxed cells are table column headers defining the content of 
each column. The second row of the table is an example of a row entry taken from the second entry of the web search 
results shown in Figure 26C. The number of links in the last two columns represent arbitrary values selected for this 
50 example. As shown in Table 2 above, in the preferred arrangement, the search engine returns to the client not only 
the ten or so most relevant document references and summaries (where relevance is determined algorithmically) as 
is typically done, but also a database containing a much larger list of references (e.g., the top 1000 documents), 
summaries, and the unweighted vector of factors. This allows very fast recompilation and display of the "most relevant" 
document references. 

55 [0053] The examples shown in Table 1 could be generated by moving a cursor appropriately between the anchors, 
"trust", "psychology" and "web". The other factors, e.g., document, recency, and popularity, would continue to affect 
the relevancy rankings as before. The cursor's movements would only affect the weights associated with the chosen 
anchors. In the preferred arrangement, the list of relevant document references and summaries is re-generated when- 
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ever the mouse button is released in a drag and drop manipulation. 

[0054] By including or excluding various anchors and moving the anchors and the position of the cursor relative to 
these anchors, a user may explore the list of retrieved document references far more efficiently than moving to each 
'next' screen of 10 references, or resubmitting the query with repetitions of some of the query terms. 
5 [0055] Although the present invention has been described in relation to particular preferred embodiments thereof, 
many variations, equivalents, modifications and other uses will become apparent to those skilled in the art. It is pre- 
ferred, therefore, that the present invention be limited not by the specific disclosure herein, but only by the appended 
claims. 

10 

Claims 

1 . An electronic display system operative to facilitate interactive graphical interface interpolation of text-based factors 
by a user, comprising: 

15 

a central processing unit, coupled to a system bus; 

a memory unit coupled to the system bus and having loaded 

therein an operating system, application programs and computer-executable instructions for: 

20 inserting a desired factor onto a first window; 

inserting node terms onto a second window by, for each node term, selecting a desired selection from a 
plurality of predetermined selections; and 

upon a cursor being dragged over the second window to a desired node term, additively applying char- 
acteristics for the desired node term to the desired factor based on a proximity of the cursor to the desired 
25 dot; 

a display unit coupled to the system bus; 

a cursor control unit arranged to provide signals to control movement of a cursor on the display unit; and 
the system bus, for linking the central processing unit, the display unit, the memory unit, and the cursor control 
30 unit. 

2. The electronic display system of claim 1 wherein the characteristics for the node terms are at least one of: 

query terms in a search query; 
35 frequency of query terms; 

distribution of query terms; and 
components in a computer simulation. 

3. The electronic display system of claim 1 wherein the electronic display system is a computer display system. 

40 

4. The electronic display system of claim 1 wherein inserting node terms further includes combining a plurality of 
desired node terms to form a compound node term. 

5. The electronic display system of claim 1 wherein a palette in a third window shown on the display unit is used for 
45 selecting a factor to be inserted onto the first window. 

6. A method for facilitating interactive, expressive, three dimensional textual interpolation on an electronic display 
system by a user, comprising the steps of : 

50 inserting a desired factor onto a first window; 

inserting node terms onto a second window by, for each node term, selecting a desired selection from a plurality 
of preselected selections; and 

dragging a cursor over the second window to a desired node term wherein characteristics for the desired node 
term are additively applied to the desired character/image/query term based on a proximity of the cursor to 
55 the desired node term. 

7. The method of claim 6 wherein the characteristics for the node terms are at least one of: 
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query terms in a search query; 
frequency of query terms; 
distribution of query terms; and 
components in a computer simulation. 

5 

8. The method of claim 6 wherein the electronic display system is a computer display system. 

9. The method of claim 6 wherein inserting node terms further includes combining a plurality of desired node terms 
to form a compound node term. 

10 

10. The method of claim 6 wherein a palette in a third window displayed on the display unit is used for selecting a 
desired factor to be inserted onto the first window. 

11. A computer-readable medium having computer-readable instructions for providing an interactive graphical user 
15 interface for interpolation of text-based factors by a user, wherein the computer-executable instructions include: 

inserting a desired factor onto a first window; 

inserting node terms onto a second window by, for each node term, selecting a desired selection from a plurality 
of predetermined selections; and 
20 upon a cursor being dragged over the second window to a desired node term, additively applying character- 

istics for the desired node term to the desired factor based on a proximity of the cursor to the desired node term. 

12. The computer-readable medium of claim 11 wherein the characteristics for the node terms are at least one of: 

25 query terms in a search query; 

frequency of query terms; 
distribution of query terms; and 
components in a computer simulation. 

30 13. The computer-readable medium of claim 11 wherein inserting node terms further includes combining a plurality of 
desired node terms to form a compound node term. 

14. The computer-readable medium of claim 11 wherein a palette in a third window is utilized for selecting a desired 
factor to be inserted onto the first window. 

35 

15. A method for facilitating textual interpolation on an electronic display system using a graphical user interface, 
comprising the steps of: 

dragging a pointer over an arrangement of a plurality of node terms in a target window wherein each text- 
40 based node term represents a displacement of a state of an object from a base state; and 

redrawing/updating the base state of the object in a display window in accordance with the proximity of the 
pointer to the text-based node terms as the pointer is dragged over the target window. 

16. The method of claim 15 wherein positions of the plurality of text-based node terms in the target window are set by 
45 the user. 

17. The method of claim 1 6 wherein the user uses the pointer to position the plurality of text-based node terms. 

18. The method of claim 15 wherein each target window has a predetermined area of influence that is used to deter- 
50 mine, based on a position of the pointer, the displacement to be applied to the object. 

19. The method of claim 15 wherein the state of the object is redrawn/updated by putting an object into a default base 
state when a position of the pointer changes, then applying the text-based node terms to the object based on a 
weighting of each text-based node term, wherein the weighting is calculated based on the displacement of the 

55 pointer from the text-based node term. 

20. The method of claim 15 wherein each redrawing/updating of the base state of the object is recorded to provide a 
textual update path. 
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21. The method of claim 20 wherein the textual update path is editable. 

22. The method of claim 15 wherein multiple node terms with individual weightings are applied simultaneously. 

23. A computer-readable medium having computer-executable instructions for facilitating interactive textual interpo- 
lation using a graphical user interface, wherein the computer-executable instructions include: 

dragging a pointer over an arrangement of a plurality of text-based node terms in a target window wherein 
each text-based node term represents a displacement of a state of an object from a base state; and 
redrawing/updating the base state of the object in a display window in accordance with the proximity of the 
pointer to the text -based node terms as the pointer is dragged over the target window. 

24. The computer-readable medium of claim 23 wherein positions of the plurality of text-based node terms in the target 
window are set by the user. 

25. The computer-readable medium of claim 24 wherein the user uses the pointer to position the plurality of node terms. 

26. The computer-readable medium of claim 23 wherein each target window has a predetermined area of influence 
that is used to determine, based on a position of the pointer, the displacement to be applied to the object. 

27. The computer-readable medium of claim 23 wherein the state of the object is redrawn/updated by putting an object 
into a default base state when a position of the pointer changes, then applying the text-based node terms to the 

-object based on a weighting of each text-based node term, wherein-the weighting is calculated based on the 
displacement of the pointer from the text-based node term. 

28. The computer-readable medium of claim 23 wherein each redrawing/updating of the base state of the object is 
recorded to provide a textual update path. 

29. The computer-readable medium of claim 28 wherein the textual update path is editable. 

30. The computer-readable medium of claim 23 wherein multiple text-based node terms with individual weightings are 
applied simultaneously. 
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FIG. 6A 




20 



EP 1 193 623 A2 




21 



EP 1 193 623 A2 
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FIG. 1 0A 
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FIG. 21 A 
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